Occlusion Handling using Semantic Segmentation and Visibility-Based Rendering for Mixed Reality

Real-time occlusion handling is a major problem in outdoor mixed reality system because it requires great computational cost mainly due to the complexity of the scene. Using only segmentation, it is difficult to accurately render a virtual object occluded by complex objects such as trees, bushes etc. In this paper, we propose a novel occlusion handling method for real-time, outdoor, and omni-directional mixed reality system using only the information from a monocular image sequence. We first present a semantic segmentation scheme for predicting the amount of visibility for different type of objects in the scene. We also simultaneously calculate a foreground probability map using depth estimation derived from optical flow. Finally, we combine the segmentation result and the probability map to render the computer generated object and the real scene using a visibility-based rendering method. Our results show great improvement in handling occlusions compared to existing blending based methods

Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching

We propose a non-learning depth completion method for a sparse depth map captured using a light detection and ranging (LiDAR) sensor guided by a pair of stereo images. Generally, conventional stereo-aided depth completion methods have two limiations. (i) They assume the given sparse depth map is accurately aligned to the input image, whereas the alignment is difficult to achieve in practice. (ii) They have limited accuracy in the long range because the depth is estimated by pixel disparity. To solve the abovementioned limitations, we propose selective stereo matching (SSM) that searches the most appropriate depth value for each image pixel from its neighborly projected LiDAR points based on an energy minimization framework. This depth selection approach can handle any type of mis-projection. Moreover, SSM has an advantage in terms of long-range depth accuracy because it directly uses the LiDAR measurement rather than the depth acquired from the stereo. SSM is a discrete process; thus, we apply variational smoothing with binary anisotropic diffusion tensor (B-ADT) to generate a continuous depth map while preserving depth discontinuity across object boundaries. Experimentally, compared with the previous state-of-the-art stereo-aided depth completion, the proposed method reduced the mean absolute error (MAE) of the depth estimation to 0.65 times and demonstrated approximately twice more accurate estimation in the long range. Moreover, under various LiDAR-camera calibration errors, the proposed method reduced the depth estimation MAE to 0.34-0.93 times from previous depth completion methods.Comment: 15 pages, 13 figure

Combined Influences of Gradual Changes in Room Temperature and Light around Dusk and Dawn on Circadian Rhythms of Core Temperature, Urinary 6-Hydroxymelatonin Sulfate and Waking Sensation Just after Rising

The present experiment aimed at knowing how a gradual changes of room temperature (Ta) and light in the evening and early morning could influence circadian rhythms of core temperature (Tcore), skin temperatures, urinary 6-hydroxymelatonin sulfate and waking sensation just after rising in humans. Two kinds of room environment were provided for each participant: 1) Constant room temperature (Ta) of 27 °C over the 24 h and LD-rectangular light change with abrupt decreasing from 3,000 lx to100 lx at 1800,abrupt increasing from 0 lx to 3,000 lx at 0700. 2) Cyclic changes of Ta and with gradual decrease from 3,000 lx to 100 lx onset at 1700 (twilight period about 2 h), with gradual increasing from 0 lx to 3,000 lx onset at 0500 (about 2 h). Main results are summarized as follows: 1) Circadian rhythms of nadir in the core temperature (Tcore) significantly advanced earlier under the influence of gradual changes of Ta and light than no gradual changes of Ta and light. 2) Nocturnal fall of Tcore and morning rise of Tcore were greater and quicker, respectively, under the influence of gradual changes of Ta and light than no gradual changes of Ta and light. 3) Urinary 6-hydroxymelatonin sulfate during nocturnal sleep was significantly greater under the influence of gradual changes of Ta and light. 4) Waking sensation just after rising was significantly better under the influence of gradual changes of Ta and light. We discussed these findings in terms of circadian and thermoregulatory physiology

Prenatal Torsion of Radial Polydactyly: A Gangrenous Mass at the Base of the Thumb

A patient was born with a mass at the base of the thumb approximately 1.5 cm in diameter on the radial side of the fingers. The mass had globular swelling filled with hemorrhagic fluid and was dark red. X-rays and histology of the excised specimen suggested the diagnosis of gangrene and torsion of polydactyly. Prenatal torsion of polydactyly is not a common occurrence; moreover, prenatal torsion of polydactyly has only been found in ulnar polydactyly. Our case is a novel case of radial polydactyly that was gangrenous at birth owing to prenatal torsion. Diagnosing such a mass at the base of the thumb is important